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Abstract 

An approach is proposed which, given a family of hnearly independent functions, constructs the 
appropriate bi-orthogonal set so as to represent the orthogonal projector operator onto the correspond- 
ing subspace. The procedure evolves iteratively and it is endowed with the following properties: i) 
it yields the desired bi-orthogonal functions avoiding the need of inverse operations ii) it allows to 
quickly update a whole family of bi-orthogonal functions each time that a new member is introduced 
in the given set. The approach is of particular relevance to the approximation problem arising when 
a function is to be represented as a finite linear superposition of non orthogonal waveforms. 



I. Introduction 

Mathematical modelling of physical systems often involves the representation of functions as a linear 
superposition of waveforms. The formal theory of such representations is concerned with completeness 
and expansion properties of the corresponding waveforms. The most general setting for this purpose 
being provided by the frame theory. Although introduced early in the context of non-harmonic Fourier 
analysis Q it is only recently that the theory of frames has received great interest, as it was adopted 
to study wavelets and Gabor-like functions IH, @, ||5|, @. Frames are characterised by being, in gen- 
eral, redundant (overcomplete) . This implies that for a given frame the removal of some of its elements 
can still leave a frame. Otherwise the frame is said to be an exact one. In Hilbert spaces, to which our 
considerations will be restricted, an exact frame is equivalent to a Riesz basis. A further requirement 
on the nullity of the inner product between different elements amounts to an orthogonal basis. 



The representation of functions within the frame structure has been the subject of much work during 
the last decade §, §, |], §, |], [|, @, Q, [0, H, ig. On the other hand, the problem 
of approximating functions through the linear superposition of a finite number of frame elements has 
been addressed less extensively (some discussion in the context of particular applications can be found 
in 1 15 1, fl^). Here we would like to contribute to such a discussion by focusing on an approxi- 
mation problem arising by considering a finite number of elements out of a Riesz basis. We believe 
it to be important to stress that approximations arising by truncating expansions given in terms of 
orthogonal waveforms and those given in terms of non-orthogonal ones are of quite different nature. 
By truncating an orthogonal expansion one obtains the approximation in the corresponding subspace 
which minimises the distance to the exact function under consideration. Nevertheless, an incomplete 
non-orthogonal expansion does not guarantee an approximation of the same quality. If one wishes to 
obtain an optimal approximation in the minimum distance sense by means of a non-orthogonal expan- 
sion, the coefficients of the corresponding linear combination should be re-calculated. In other words, 
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while by truncating terms in the representation of an identity operator which is given by orthogonal 
waveforms one obtains an orthogonal projector, an equivalent procedure in the non-orthogonal case 
does not leave an orthogonal projector. 

Representations of the identity operator in term of non-orthogonal bases involves bi-orthogonal 
ones. Indeed, if the sequence a„ ; n = 1, . . . , oo is a Riesz basis for a Hilbert space, then there exists 
a reciprocal basis d„ ; n = 1, . . . , oo which is bi-orthogonal to the former, i.e. {am\oin) = ^m,n- The 
identity operator in the corresponding Hilbert space can, thereby, be expressed as 

oo 

/=^q„(q„|.) (1) 

n=l 

where (a„| .) indicates that / acts by performing inner products, as in // = Yl'^=i'^n{ctn\f) ■ Now, 
if the sum in (||) is truncated up to terms, then the approximation of a function / that we obtain 
by computing Y^n=i ^n{cin\f) is not the best approximation of / that one can obtain as a linear su- 
perposition of elements a„ ; n = 1, . . . , A^. This is tantamount to stating that the operator that is 
obtained by truncating the sum in (||) up to N terms is not the orthogonal projector onto the subspace 
spanned by the elements a„ ; n = 1, . . . , A^. Indeed, as will be discussed in subsequent sections, the 
bi-orthogonality condition does not yield, per se, orthogonal projections. In order to construct an 
orthogonal projector operator the appropriate bi-orthogonal functions need to be computed. 

Given a set of A'^ linearly independent functions spanning a subspace, say Sat, our goal is to develop 
an effective procedure for constructing the corresponding bi-orthogonal functions giving rise to the 
orthogonal projector onto Sm- An advantage of the algorithm we propose here being its capability of 
building up the desired bi-orthogonal functions avoiding the need of inverting operators. This feature 
allows to quickly adapt the whole set of bi-orthogonal functions each time that the corresponding sub- 
space is enlarged by adding a new element to the given initial set. We assume that all these elements 
are linearly independent, however, in the event that this hypothesis fails to be true, the proposed 
algorithm provides a criterion for disregarding linearly dependent elements. Thereby the algorithm 
itself can be used for selecting subsets of linearly independent functions. 

The paper is organised as follows: Section II introduces the notation and discusses the problem 
of constructing orthogonal projections by means of non-orthogonal waveforms. In section III an 
iterative procedure to compute bi-orthogonal functions giving rise to orthogonal projector operators 
is advanced. The algorithm is illustrated by computing the bi-orthogonal functions amounting to the 
orthogonal projector operator onto a subspace generated by a few Mexican hat wavelets. 
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II. Notation, background and preliminary considerations 
A. Bi- orthogonal expansions 



Adopting Dirac's vector notation we represent an element / of a Hilbert space as a vector 

I/) and its dual as (/|. Given a set of (5-normalised continuous orthogonal vectors {\t) ; — oo < t < 
oo ; {t\t') = 6{t — t')}, the unity operator in Ti is expressed 

/„= hm r \t){t\dt. (2) 

Thus, for all |/) and \g) G TL, by inserting /^^ in (/|(7), i.e, 

(/|/^b)= lim r {f\t){t\g) dt (3) 

one is led to a representation of TC in terms of the space of square integrable functions, with {t\g) = g{t) 



and {g\t) = {t\g) = g{t), where g{t) indicates the complex conjugate of g{t). 

Let vectors |a„) £ TC ; n = 1, . . . , oo be a Riesz Basis for TC. Then there exists a reciprocal basis 
|q!„) ; n = 1, . . . , oo for 7^ to which the former basis is bi-orthogonal i.e., {a.n\am) = Sn,m- These two 
bases amount to a representation of the identity operator as given by: 

oo oo 

i = ^ l«n)("n| = X! I«n)("n|, (4) 
n=l n=l 

SO that every \ f) £ TC can be expanded in the form 

oo oo 

I/) = \an){an\f) = E l«n)(a„|/). (5) 

n=l n=l 

The statements above concerning bi-orthogonal bases are well established results |2|. Nevertheless, the 
corresponding approximations arising by considering a finite number of vectors |a„) ; re = 1, . . . , A'^ 
have thusfar received much less consideration. It is important to stress that some properties holding 
on truncation of an orthogonal basis, no longer hold in the bi-orthogonal case. Let us consider for 
instance that the sum in (Q) is truncated up to N terms. Thus, rather than a representation of the 
identity operator in TC we obtain an operator, Q say, given by 

N 

Q = Y\an){an\. (6) 

n=l 

Unlike the case involving an orthogonal basis, Q is not the orthogonal projector operator onto the 
subspace spanned by N vectors |a„) ; re = 1,...,A^. The bi-orthogonality property of families 
; re = 1, . . . , and ; re = 1, . . . , A^ implies that = Q, hence Q is indeed a projector. 
However, (the adjoint of Q) is not equal to Q and therefore Q is not an orthogonal projector. As 

draft 



a consequence, the approximation |/jv) of |/) that we obtain from: 

N 

\fN)=Qf=Y.\^n){an\f) (7) 
n=l 

is not the best approximation of |/) that can be obtained as a hnear superposition of vectors \an)- 
This being actually a major difference between orthogonal and bi-orthogonal expansions. Whilst the 
condition of orthogonality yields orthogonal projections by truncating expansions, the bi-orthogonality 
condition, per se, does not guarantee projections of such a nature. If one wishes to obtain orthogonal 
projections by means of bi-orthogonal families, then bi-orthogonal vectors specially devised for such 
a purpose must be constructed. 

B. Building Orthogonal Projections 

Let us consider a subspace, Sjy of 7i spanned by N linearly independent vectors \an) ', n = 1, . . . , N. 
Hence, every \ f) £ 7i can be decomposed in the form 

\f) = \fsj + \fs^), (8) 

where I/sn) ^ ^N, i-e., Usm) — J2n=i '^n\c(n) for some coefficients Cn ; n = 1, . . . , N . Now, in order 
for 1/5^) to be the orthogonal projection of |/) onto Sn we should require that \fg±) belongs to Sj^, 
the orthogonal complement of Sjy in 7i. This implies that 1/5^) should be orthogonal to every vector 
\an) ; n = 1, . . . , N and we are thus led to the following equations: 

N 

{ak\f) = ^Cn{ak\an) ; k = l,...,N. (9) 

n=l 

Let |n) ; n = 1, . . . , be the standard basis for C^, i.e, (m|n) = 6m,n so that vector |c) G can be 
expressed as |c) = Z]^=i(™|c)|'T') = J2n=i Cn|?^) and equations (^) can be re-written as 

{ak\f) = F\c) (10) 

N 

F = J2\an){n\. (11) 

n=l 

N 

Ft = ^|n)(a„|, (12) 

n=l 

the left-hand-side of equation ( [lO|) happens to give the components of vector F'^\f) G , whereas on 
the right-hand-side we find the components of a vector F^F\c) G . Thus, these equations can be 
recast in the form 

F^\f) = F^F\c). (13) 
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where operator F is given by 



Since the adjoint F^ of F is 
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Since vectors |a„) ; n = 1, . . . , N are linearly independent operator F'^F has an inverse and |c) is 
readily obtained as 

|c) = (Ft^)-i^t|/). (14) 
The corresponding components Cn = {n\c) being 

Cn = (n|c) = (n|(Ft^)-i^t|;) = («A^|/), (15) 

with vectors {a^ \ given by 

{a^\ = {n\{F^F)-'F^. (16) 
Let us consider now the eigenvalue equation for the positive operator F^F 

F^ F\r]n) = XnlVn) n = l,...,N (17) 

where A„ > and the corresponding eigenvectors G satisfy the condition {rik\r]n) = ^k,n- 
It readily follows that the orthonormal vectors l^n) = '^j'^"^ G 7i are eigenvectors of the positive 
self-adjoint operator 

N 

G = FF^ = Y,\an){an\ (18) 

n=l 

with associated eigenvalues ; n = 1, . . . , A^. Hence, the spectral decomposition of G is 

N 

G=T. \<Pn)Xn{cpn\ (19) 
n=l 



and that of its inverse, G ^, is 



^ 1 

G~' = J2\^n)T-{cl>n\- (20) 
n=l -^"^ 



Moreover, since Range(G)= S^, the orthogonal projector operator onto Sn can be expressed as 



N 



p = G'^G = GG"^ = J2\^n){^n\ (21) 

n=l 

We are now in a position to provide an alternative form for vectors {a^ \ given in (p!6|). 
Proposition l:Vectors (a^l, which are defined as 

(a^| = (n|(Ft^)-iFt (22) 

can be expressed as: 

{a^\ = {an\G'' (23) 
Proof: Replacing (F'^F)^^ in (|l6|) by its spectral decomposition J2k=i \ Vk) {Vkl we have: 

N , 

(d^l = (n|(Ft^)-ij7t = ^(^1^,) /^,|j7t. (24) 

k=i 
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Since (r/fcl-F"!" = ^/Xk{(j)k\ and |??fc) = ^^l^fc) from (p3) we obtain: 



N , 

(a^l = Y^{n\F^\4>,) = {n\F^G-\ (25) 

k=i 

On the other hand, from (^) we gather that {n\F'^ = (q!„| and the proof is concluded □ 
Proposition 2: The reciprocal families ) ; n = 1, . . . ,N and |q„) ; n = 1, . . . , are bi-orthogonal 
families. 

Proof: It is straightforward by writing (a^| = {n\{F'^ F)~^ F'^ and \ak) = F\k). Indeed, 

(d^lofc) = {n\{F^Fr^F^F\k) = {n\k) = Sn.k^ (26) 

The next proposition shows that the families ) ; n = \, . . . ,N and \an) ', n = 1, . . . ,N provide us 
with a representation of the orthogonal projection operator onto Sn- 

Proposition 3: Vectors \a^) ; n = 1, . . . ,N and |a„) ; n = 1, . . . , amount to a representation of 
the orthogonal projection operator onto Sn as given by 

N N 

P = ^|a.)(d^|^5:|a„)(d^| (27) 

n=l n=l 



Proof: It immediately follows by using (|T^) in (21) 



N N 



P = G'G'-i = ^|a„)(a„|G'^i = ^|a„)(Q^ 

n=l ra=l 
N N 

= G'-lG'= ^G-Vn)(an| = El"n)("nl° (28) 
n=l n=l 

It is clear at this point that if we introduce a new vector |a7v+i) in the initial set, then in order 
to represent an orthogonal projector onto the enlarged subspace all the corresponding bi-orthogonal 
vectors |a^+^) ; n = 1,...,A^ + 1 must be re-calculated. The above discussion implies that the 
calculation of these vectors involves the computation of an inverse operator. Our goal is to avoid the 
need for this calculation. In the next section we propose a recursive procedure that enables us to 
compute bi-orthogonal vectors endowed with the desired properties. 

III. Recursive bi-orthogonal approach 

From the discussion of the previous section we are in a position to assume the existence of bi- 
orthogonal families of vectors in 7i giving rise to orthogonal projections on the subspace they span. 
Accordingly, given k vectors |q„) ; n = 1, . . . ,/c spanning a subspace Vk let us denote |d^) ; n = 
1, . . . , fc to the corresponding bi-orthogonal family yielding a representation of the orthogonal projector 
operator onto Vk i.e., 

k k 

A = El«n>(«n| = El«-)("n|- (29) 
n=l n=l 
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If we add now a new element to the previous family, in order to represent the orthogonal 

projector onto the the enlarged subspace V^+i, we need to compute a new bi-orthogonal family 
la^"^"^) ; n = l,...,/c + l. The operator Pfc+i in terms of the new family is: 

fc+i fc+i 

A+l = E l«n+')(«nl = E Wn){ai+\ (30) 
71=1 n=l 

Now, is the the direct sum T4+i = © |Qfc_|_i). Hence, the orthogonal projector operator onto 
Vfc_|_i can be decomposed in the fashion: 

Pk+i = Pk + Pit (31) 

where Pk is given in (|29| ) and is the orthogonal projector onto the subspace Vj^' denoting the 
orthogonal complement of in V^+i. Consequently, Vj^' is spanned by the single element iV'fc+i)) 
which is obtained by removing from |afc+i) its component in V^, i.e. 

\^pk+l) = \ak+i) - Pk\ak+i)- (32) 



Hence Pr*- is obtained from as 

lll^fc+i>ll 



and we can explicitly re-write (|3l|) in the form 



E = E l«n)(«.| + (34) 



—1 ;ri iir'/^fe+i 

or, equivalently. 



IV'fc+l)(V'fc^ 



+ 2^ )(««! = 2^ |an)("n| + -ml TTTa-- (^5) 

n=l n=l lll^fc+l/ll 



By taking the inner product of both sides of (|35| ) with Id^) ; n = 1, . . . , A; we obtain: 

|afci})(afc+i|a^) + |a^,+^) = ; n=l,...,k. (36) 

On the other hand, by taking the inner product of both sides of (|35| ) with we have 

\alXl){ak+i\iJk+i) = \i^k+i). (37) 
The last two equations yield the recurrent formula that was our goal to find: 

|a^+i) = |a^) - ; n = l,...,k 



'■'k+D 
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We show now that these vectors indeed satisfy the desired properties. The next theorem is in order. 
Theorem 1: Let \an) ; n = 1, . . . , /c + 1 be a set of linearly independent vectors spanning a subspace 
Vfc+i. Then vectors jaJi'''^) ; n = 1, . . . , k + 1 constructed as prescribed in (^), by considering iV'i) = 
|ai), satisfy the following properties: 

a) are bi-orthogonal to vectors |q„) ; n = 1, . . . , k + 1 i.e., 

b) provide a representation of the orthogonal projection operator onto Vfc+i, i.e., 

fc+l /l'+I 

A+l = E l«n)(«n+'l = H+l = E (39) 
n=l n=l 

Both the proofs of a) and b) are achieved by induction. The corresponding steps are to be found in 
Appendix A. 

Corollary 1: The coefhcients c^^^ of the linear expansion 

Ec^Vn) (40) 
n=l 

which approximates an arbitrary vector |/) G at best in a minimum distance sense, can be recursively 
obtained as: 

fc+i _ k_i~k\ \ (V'fc+il/) . , , 

\\\Vk+l)\\ 

k + l _ (V'fc + ll/) 

iiiv'.+i>ir ^ ^ 

with - 

Proof: The proof results from the fact that the unique vector in V^+i which minimises the distance 
to I/) is obtained as Pk+i\f)- Indeed, let |/') be an arbitrary vector in Vk+i and let us write it as 
If) = \f') + Pk+i\f)-Pk+i\f)- If we calculate the squared distance |||/) - |/'>|P, since (|/)-A+i|/)) 
is orthogonal to every vector in Vk+i, we have 

- I/'>IP = III/) - A+il/) + 1/') - A+il/) = III/) - A+il/)lP + III/') - A+il/)lP 



from where we gather that the distance is minimised if |/') = Pk+i\f)- Hence (41) readily follows by 
using (p^) in ( |39| ) and identifying c^"*"^ with {a^^^\f) for n = l,...,A; + in 

We illustrate the proposed approach by the following example: Let us consider A'^ = 5 elements 
\an) n = 1, ... ,5 whose functional representations are given by the following shifted Mexican hat 
wavelets 

an{t) = (t|a„) = ^-^e-(*-+i)' {1 - {t - n + if) ; n = 1, . . . , 5 (42) 

DRAFT 



10 

Figure 1 plots the wavelet ai. Figure 2 plots the functions a} (thin line), af (dotted line) and df 
(thick line) which are involved in the representation of the orthogonal projectors onto the subspaces 
spanned, respectively, by the sets {ai}, {01,02,03}, {01,02,03,04,05} 

IV. Conclusions 

An iterative procedure, for constructing bi-orthogonal functions yielding orthogonal projectors, has 
been proposed. The outcome being relevant to the approximation problem concerning the representa- 
tion of a function as a linear supposition of waveforms. The proposed method avoids the computation 
of inverse operations and allows us to update the whole set of bi-orthogonal functions when the di- 
mension of the subspace is increased. This feature is, thereby, of great importance in situations in 
which the waveforms are to be selected from a large number of possible ones ||l9|| . 
It is appropriate to stress that if the available waveforms are linearly dependent, the proposed al- 
gorithm allows one to select subsets of linearly independent ones. This is to be achieved simply by 
disregarding those vector |ofc_|_i) yielding corresponding vectors (given in (^2|)) of zero norm. 

From the above remarks it is expected that this technique should be of assistance in a broad range of 
problems concerning mathematical modelling of physical systems. 
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Appendix 
A. Proof of Theorem 1 
We show first that vectors la^"*"^) arising from 1-01) = by the recursive formula: 

|an+^) = - |||f'^^\||2 («fc+ll«n) : n = I, . . . ,k 

\\Wk+i)\\ 

\^k+ii - III,/,,, \||2 y^-^) 



yfc+1 

are bi-orthogonal to vectors \an) ; n = 1, . . . , A; + 1, i.e., they satisfy the relation 

{am\oi^^) = 5m,n n=l,...,k + l ; m = l,...,k + l (A.2) 

For k = the relation holds because |q;}) = and therefore (ai|d];) = = 1. Assuming 

that for A; + 1 = / it is true that 

{am\al) = 5m,n ] n = l,...,l ; m = l,...,l (A.3) 

we shall prove that 

{am\a^+^) = Sm,n ] n = 1, ...,/ + 1 ; m =!,...,/ + !. (A.4) 

In order to prove this we need to discriminate between four different situations concerning the values 
of indices m and n. 

l)m = 1, . . . ,1 and n = 1, . . . ,1 
For this situation {am\i^i+i) = 0. Hence, using ( [A.l| ) and (|A.3| ) we have: 

/ I («m|V'i+l)/ 



Ui/ 

(A.5) 



II)m = 1 + 1 and re = 1, . . . , Z 
Thus 

l~i+l\ u («Z+l|V'f+l) / |~«\ 



«+i;ii^ 

W+i "n) TTTl rTl2 = 0' vA.6) 



since = ("i+il'^i+i)- 



III)m = 1 + 1 and re = / + 1 
This implies 
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IV)?7i = and n = I + 1 

In this case {am\tpi+i) = 0. Hence 



(a \a'+^) - i^IIl\^mL - (A 



From I) II) III) and IV) we conclude that 

(am|a^+^) = (5m,n ; n=l,...,A; + l ; m = l,...,A; + l 

Now we prove that vectors ;n = l,...,A;+l given in ( [A.l| ) provide a representation of the 

orthogonal projection operator onto Vk+i as given by: 

k-\-l /l'+I 

P,+l = E l«n+')(«nl = = E l«n)(«n+'l (^-9) 

n=l n=l 

We prove that J2n=i \'^n){(in'^^\ is the orthogonal projector onto 14+1 by proving 
OEnil l«n)(«^^|5) = |5) ; V b) G T4+1 and 
ii)E^;i |a„)(a^i|5^) = 0; V \g^) G y,-^,. 



i)is actually a consequence of the bi-orthogonality condition ( [A.2 ). Because if \g) G V^+i it can be 



expressed as \g) = J2n=\ Cn\ctn) for some coefficients c„ ; n = 1, . . . , A; + 1, hence X]n=i \'^n){ctn^^\9) = 

Entlcnlan) = Iff)- 

We prove ii) by induction. For A: + 1 = 1 we have Alff"*") = ^"Mi^"\lf2 ^ = 0; V \g-^) G Vf^, since 
(ailff^) = for ah 1^^) G T/j^. 
Let us assume that 

k 

P,\g^) = Y,\an){at\9^)=0 ; y \g^) g V,^ (A.IO) 

n=l 

and prove that it is then true that 

fc+i 

Pk+i\g^) = J2\(^n){atV) = o ; y\g^)^Vk\i- (A.ll) 

n=l 

Indeed, 

E*^"*""^ I \/-fc+li ±\ I \ (V'fc+lls'"'") , 5 I ±\ 5 I \(V'fc+lb"'") n 1o^ 
J<'Ma„ I, ) = K,,)p^+ftl9 )-nK+i)p^=0. (A.12) 

The zero value of the above equation holds from the fact that {an\g'^) = 0; n = l,...,/c + l for all 
\g^) G Ffc^i and Pk\g^) = by hypothesis (|AlO| ). 



Since Pfc+i = Z]n=i l'^n)(Q^n^^| is an orthogonal projector it is true that Pk+i = pI^i = J2n=\ \oin^^){(^n\ 
so that the proof is completed □ 
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Figure 2: Functions C(\ (thin line), df (dotted line) and af (thick line). 
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Figure Captions 

Figure 1: Mexican hat wavelet ai. 

Figure 2: Functions a\ (thin hne), af (dotted hne) and af (thick hne). 
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